Skip to content

Comments

Fix DS/QWEN Example#2354

Merged
chensuyue merged 6 commits intomasterfrom
fix-example
Dec 12, 2025
Merged

Fix DS/QWEN Example#2354
chensuyue merged 6 commits intomasterfrom
fix-example

Conversation

@yiliu30
Copy link
Contributor

@yiliu30 yiliu30 commented Dec 8, 2025 •
edited
Loading

User description

Signed-off-by: yiliu30 yi4.liu@intel.com

Type of Change

  • Remove the limit 64
  • Fix QWen OoM

@yiliu30
unset limit
a2257bc 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
@yiliu30 yiliu30 changed the title unset limit Fix DS/QWEN Example Dec 8, 2025
@PRAgent4INC PRAgent4INC changed the title Fix DS/QWEN Example unset limit Dec 8, 2025
@PRAgent4INC
Copy link
Collaborator

PRAgent4INC commented Dec 8, 2025

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Possible Issue

Removing the --limit parameter might lead to unexpected behavior if the script is not designed to handle an unlimited number of samples. Ensure that the script can handle the increased load without issues.

 
lm_eval --model vllm \
  --model_args "pretrained=${MODEL_PATH},tensor_parallel_size=${TP_SIZE},max_model_len=8192,max_num_batched_tokens=32768,max_num_seqs=128,add_bos_token=True,gpu_memory_utilization=0.8,dtype=bfloat16,max_gen_toks=2048,enable_prefix_caching=False" \
  --tasks $TASK_NAME \
  --batch_size $BATCH_SIZE \
  --log_samples \
  --seed 42 \
  --output_path ${OUTPUT_DIR} \
  --show_config 2>&1 | tee ${OUTPUT_DIR}/log.txt
Possible Issue

Removing the --limit parameter might lead to unexpected behavior if the script is not designed to handle an unlimited number of samples. Ensure that the script can handle the increased load without issues.

 
lm_eval --model vllm \
  --model_args "pretrained=${MODEL_PATH},tensor_parallel_size=${TP_SIZE},max_model_len=8192,max_num_batched_tokens=32768,max_num_seqs=128,add_bos_token=True,gpu_memory_utilization=0.8,dtype=bfloat16,max_gen_toks=2048,enable_prefix_caching=False" \
  --tasks $TASK_NAME \
  --batch_size $BATCH_SIZE \
  --log_samples \
  --seed 42 \
  --output_path ${OUTPUT_DIR} \
  --show_config 2>&1 | tee ${OUTPUT_DIR}/log.txt

@PRAgent4INC
Copy link
Collaborator

PRAgent4INC commented Dec 8, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Verify --limit removal impact

Ensure that the absence of --limit does not lead to excessive resource usage or
unintended behavior.

examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/deepseek/run_evaluation.sh [115-117]

+--batch_size $BATCH_SIZE \
+--log_samples \
+--seed 42 \
 
-
Suggestion importance[1-10]: 5

__

Why: The suggestion asks to verify the impact of removing --limit, which is a reasonable concern. However, it does not propose any concrete changes or improvements, so the score is moderate.

Low

@yiliu30 yiliu30 changed the title unset limit Fix DS/QWEN Example Dec 8, 2025
@yiliu30
fix qwen mxfp4
4a71563 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
@yiliu30
Merge branch 'master' into fix-example
4b1a6e7
@yiliu30
disable reload
3d47865 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
@yiliu30
use bf16 ds
9903c39 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
@yiliu30
update
6a77ccd 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
@chensuyue chensuyue merged commit 5864c7a into master Dec 12, 2025
14 checks passed
@chensuyue chensuyue deleted the fix-example branch December 12, 2025 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chensuyue chensuyue chensuyue approved these changes

Assignees

No one assigned

Labels

Review effort 2/5

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yiliu30 @PRAgent4INC @chensuyue